DOCUMENT RESUME 



ED 447 154 



TM 031 969 



AUTHOR 

TITLE 

PUB DATE 
NOTE 
PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



Ediger, Marlow 

Qualitative Assessment versus Measurement of Student 
Achievement . 

2000 - 00-00 

9p. 

Opinion Papers (120) 

MF01/PC01 Plus Postage. 

Elementary Secondary Education; Measurement Techniques; 

* Performance Based Assessment; Portfolio Assessment; 
*Qualitative Research; Reliability; *Science Achievement; 
^Student Evaluation; Test Construction; Validity 



ABSTRACT 



This paper focuses on how best to evaluate student 
achievement in science. Many issues such as issues of test construction, 
validity, and reliability, and how to measure hands-on achievement must be 
considered. Qualitative evaluations are harder to explain, but can yield 
valuable information. The use of portfolios is an example of a technique that 
reveals much about student achievement. However, there are problems with 
portfolio use, including those of reliability, content, and scoring. 
Performance assessments in science can relate performance objectives to 
student achievement. Science teachers need to study acceptable new trends in 
teaching and learning. They must work to engage students in active learning 
and choose assessments that measure chosen objectives. (SLD) 
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QUALITATIVE ASSESSMENT VERSUS MEASUREMENT 
OF STUDENT SCIENCE ACHIEVEMENT 

Much emphasis in educational circles and in society is discussed 
on how best to reveal learner achievement. This paper will focus on 
evaluating science achievement of students in the public schools. How 
can it be shown accurately and fairly how well learners are doing in the 
science curriculum? Toward one end of the continuum, there are 
advocates who want measurable results from students. Thus percentiles, 
standard deviations, quartiie deviations, and grade equivalents, among 
others, convey in a single numeral, the achievement of students in 
science. Numerical results are easier to understand by the lay public as 
compared to qualitative evaluations. Thus from taking a standardized or 
criterion referenced test (CRT), a student is on the 60th percentile, a 
single numeral provides Information desired by the lay public. Many 
people in society take the test results as being accurate and carved in 
stone (Ediger, 2000, 503-505). 

issues in Quantitative Measurement in Science 

There are numerous issues involved in viewing a numeral to show 
how well students are doing in science. The following are selected 
issues: 

1. how well do test items in a science reveal a representative 
sampling of what can be measured? if test items favor one school or 
school district’s students more than other learners, then the sample of 
test items favors a certain population of students who have had more 
opportunities to learn what is tested on than do the others. 

2. how well was the test constructed n terms of validity? Thus 
students must have had opportunities to study related concepts and 
generalizations on the test. Compare that with students who have not 
studied any lessons or units that pertain to selected items being tested. 

3. how well was the test pilot studied to determine reliability? A 
good test then measures consistently, be it in test/retest, alternate 
forms, and/or equivalent forms reliability. Consistency in reliability 
should make for the same test score, if in a pilot study, a student 
receives the same test results when equivalent forms are being used, or 
if the test is taken over again as in test/retest reliability. 

4. how well are test items related to each other? in a well 
constructed test, according to test specialists, there should be no 
relationship of one test item to another so that clues are not given in 
taking a multiple choice test. In actual teaching and learning situations, 
however, science curriculum experts recommend that learners in 
science perceive facts, concepts, and generalizations as being related, 
when feasible, isolated subject matter in science is more difficult to 
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remember. For example, from a science experiment, students should be 
able to relate knowledge to achieve a good summary or a generalization. 

5. how well can tests measure hands on approaches in the science 
curriculum? Hands on approaches stress students being actively 
involved in making observations of science phenomenon, conducting 
experiments and demonstrations, problem solving activities, among 
others. Tests usually are paper/pencii responses by learners to muitipie 
choice test items. In the former learning situation, students are actively 
engaged in learning whereas in test taking, students respond to muitipie 
choice items and tend to be passive individuals (Ediger, 1999,37-46). 

Qualitative Evaluation in Science Achievement 

Qualitative evaluation in science achievement has made 
considerable input into evaluating leaner progress in science. The use of 
portfolios is one example. Portfolios emphasize products and processes 
of students to be an inherent part of the portfolio. The student here has 
considerable chances for input in terms of what should go into a 
portfolio. Thus, the student with teacher guidance develops his/her own 
portfolio. Contents in the portfolio might well be the following: 

1. written reports of science experiments performed by the student 
from an ongoing iesson or unit of study in thematic science. 

2. cassette recordings of oral book reports. 

3. drawings, written discourse, and diagrams of science content 
studied. 

4. a video-tape of cooperative learning activities involving the 
student. 

5. art products pertaining to science content studied, such as 
illustrations drawn of dinosaurs and murals of the different eras in 
geological history. 

6. seif evaluation completed in terms of desirable criteria 
developed. 

7. construction experiences, such as developing a project for the 
oncoming science fair. 

8. making papier mache’ models such as prehistoric animals. 

9. an outline of subject matter read from the basal science 
textbook. 

10. snapshots of dioramas made of prehistoric fauna and flora 
(Ediger, 2000, 10-12). 

in assessing the worth of a portfolio, much more of qualitative 
information as compared to quantitative results as in a single percentile 
will be in evidence to indicate learner results. For qualitative assessment 
a rubric needs to be developed. The results from rubric evaluation will 
generally indicate five levels from “excellent” to “needs improvement” 
to appraise separate parts, such as for written work or for the total 
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portfolio. The reason for emphasizing separate parts are the following: 

1. written work has quite different standards which need to be 
used for assessment as compared to a completed mural in science. 

2. written work stresses the use of verbal/linguistic intelligence 
whereas a completed mural emphasizes artistic intelligence. 

There are, however, qualitative criteria that may be used to 
assess the entire portfolio which cut across project lines, such as 

1. neatness 

2. accuracy 

3. creativity 

4. thoroughness 

5. promptness and dependability. 

Perhaps, both sets of criteria may be used such as those which 
are project specific as well as those that relate to all completed projects 
in a portfolio. 

Portfolio assessment has its own problems, as does the 
measurement movement. Problems on portfolio assessment include the 
following: 

1. portfolios may be very voluminous and become too time 
consuming to appraise. 

2. interscorer reliability may be a problem when two or more 
assess a portfolio, let alone assessing many portfolios in a class or 
school. 

3. interested, responsible people who desire to study student 
achievement in science through portfolio approaches may find it difficult 
to notice progress and achievement by viewing an adequate number. A 
single statistic such as a percentile provides quick, but limited, data on 
how well students are achieving. A further problem is to know what to 
look for within the many pages of a portfolio and what specifically 
relates directly to the achievement of the school’s objectives. 

4. it presents a problem in terms of assessment, since portfolios 
cannot be machine scored as can statewide or locally administered tests. 
To have teachers assist each other to assess portfolios takes away 
valuable time for planning for science instruction and adds to the work 
load of teachers. 

5. validity of portfolio results are more difficult to determine as 
compared to the testing and measurement movement, since many entries 
need to be read to ascertain validity in the science curriculum (Ediger, 
2000, 22-31). 

Portfolios used to ascertain since achievement of learners do 

provide 
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1. contextual information of how well students are doing on a daily 
basis within ongoing lessons. 

2. feedback to the teacher as to what needs additional emphasis in 
science instruction. The products and processes inherent in a portfolio 
might well assist teachers to have a well planned sequential program of 
science instruction. 

3. diagnosis and remediation teaching information for each 
student is inherent and is possible with recorded information on how 
well a student is doing specifically in science from portfolio information. 

A test score does not provide this useful information. 

4. opportunities for the teacher to provide sequential learnings in 
science for students and thus provide chances for learners to experience 
good sequence or order of experiences in the science curriculum. 

5. content might well be presented in a relationship of ideas form 
rather than being isolated items such as in a testing situation. According 
to testing and measurement experts, a test item should not provide any 
information pertaining to any other items. This fragments and isolates 
information rather than showing a relationship of ideas in science 
subject matter (Ediger, 2000, 75-79). 

Performance Objectives in Science 

Performance objectives are commonly used in providing guidance 
to science teachers in terms of what students are to learn. Related to 
these objectives are performance assessments. Performance objectives 
and assessments emphasize student learn by doing. The concept of 
“learning by doing” harmonizes well with a hands on approach in 
teaching and learning situations. Performance assessments then 
provide opportunities for science teachers to observe and hear 
explanations from students pertaining to what has been learned in 
ongoing lessons and units of study. Students are heavily involved when 
they are learning by doing as compared to being tested using 
paper/pencii tests. Observing student performance in a science 
experiment/demonstration takes more time than students responding to 
paper/pencii tests on the state or local levels of education, in 
performance objectives, the science teacher needs to observe student 
qualitative achievement in the following areas: 

1. how well the learner is able to identify relevant problems areas. 

2. how well the students is able to clarify the problem in terms of 
meaningful learning. 

3. how well the student is able to plan for solving the problem. 

4. how well the student uses a variety of valid activities and 
experiences to develop an hypothesis or tentative answer to the 
identified problem. 

5. how well the student is able to test the hypothesis in a doing 
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approach. 

6. how well the student is doing in modifying the hypothesis if 
need be. 

7. how well the student deals with modifying an original 
hypothesis. 

8. how well the student does in assessing an original or modified 
hypothesis. 

9. how well the learner is able to clearly write up a science 
experiment/demonstration. 

10. how well the student performs in orally communicating results of 
the experiment/demonstration as well as. how well the student is able to 
listen carefully to the thinking of others (Ediger, 2000, 33-34). 



Performance objectives may be achieved by students individually 
or in a cooperative learning situation (See Gardner, 1993 for a 
discussion on these two intelligence — intrapersonai and interpersonal). 

The changing emphases in teaching science can be noticed with 
the following National Science Education Standards (National Research 
Council, 1996): 



LESS EMPHASIS ON 
Assessing what is easily measured 

Assessing discrete knowledge 



Assessing scientific knowledge 

Assessing to learn what students 
do not know 



Assessing only achievement 



End of term assessments 
by teachers 

Development of external assessments 
by measurement experts alone 



MORE EMPHASIS ON 
Assessing what is most highly 
prized. 

Assessing rich, well structured 
knowledge. 

Assessing scientific 
understanding and reasoning. 
Assessing to learn what 
students do understand 



Assessing achievement and 
opportunity to learn 

Students engaged in ongoing 
assessment of their work and 
that of others. 

Teachers involved in the 
development of external 
measurements. 



Science teachers need to study acceptable new trends in teaching 
and learning. Those deemed to be of high quality in nature as well as 
those based on sound research should be assessed and implemented. 
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Psychology of Teaching Science 

There are selected principles of learning from educational 
psychology which need to be implemented in the science curriculum. 
Each of the following principles of learning may be qualitatively 
assessed as to science teacher achievement of these ends. 

Thus, teachers need to engage students in active learning. Learners 
need to be actively involved in ongoing lessons and units of study. 

Being a passive recipient of knowledge has no role to play for students 
in science. Students need to participate fully in planning and 
implementing experiments and demonstrations. Hands on approaches in 
learning means that students are actively engaged in learning. 

Students need to be ready for new objectives to be achieved. 
Background knowledge and skills need to be in the repertoire of students 
so that they may benefit optimally from new concepts, facts, and 
generalizations to be achieved. Readiness for student learning is a vital 
principle of learning to stress in the science curriculum. 

Third, students need to experience objectives which are 
challenging and yet achievable, if the new objectives are too easy 
boredom may set in . Toward the other end of the continuum, if 
objectives are excessively complex, then frustration might well be an 
end result. High expectations for learner achievement are important if 
each expectation is realistic and encourages student sequential 
progress in science. 

Fourth, the interests of students need thorough consideration when 
developing the science curriculum. If state mandated objectives are 
stressed, the science teacher and his/her students may plan learning 
opportunities to achieve these ends. At other times, the teacher may use 
explicit activities and experiences which stimulate learner interests in 
the science curriculum. 

Fifth, the science teacher needs to assist learners to perceive 
purpose in ongoing lessons and units of study in science. Thus, reasons 
for learning are important. A few minutes spent in guiding learners to 
perceive purpose in new learnings to be achieved is time well spent. A 
deductive approach may be used whereby the science teacher explains 
the importance to students in achieving the new objective(s). Also, an 
inductive procedure may be implemented, such as the science teacher 
discussing with learners the importance of achieving the new objectives. 
The latter procedure stresses considerable student input in arriving at 
purpose for studying the new facts, concepts, and generalizations to be 
encountered. 

The psychology of learning needs to be followed by science 
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teachers so that students individually may iearn as much s possible in 
science. Qualitative assessment is necessary to ascertain if the 
psychology of learning is being emphasized in the ongoing science 
curriculum. By using a checklist or rating scale, the quality of instruction 
may be evaluated. Each of the above named criteria pertaining to the 
psychology of teaching science may then be appraised. Self appraisal 
by the science teacher may be used and/or a coiieague may assist in 
using the checklist or rating scale to appraise teacher performance 
(Ediger, 2000, 101-103). 



References 

Ediger, Marlow (2000), Choosing Evaluation Procedures,” 
Education. 120 3), 503-505. 

Ediger, Marlow (1999), “Scope and Sequence in Science,” School 
Science. 37 (4), 37-46. 

Ediger, Marlow (2000), “Project Methods in Science,” Becoming. 

11 ( 2 ), 10 - 12 . 

Ediger, Marlow (2000), “Portfolios and the Middle School Pupil,” 
Michigan Middle School Journal. 22-31. 

Ediger, Marlow (2000), “Improving the Health Education 
Curriculum. Experiments in Education. 28 (5), 75-79. 

Ediger, Marlow (2000), “Reading Achievement in Science,” 
Spectrum. 26 (1), 33-34. 

Ediger, Marlow (2000), “ingredients in the Curriculum,” The 
Educational Review. 106 (7), 101-103. 

Gardner, Howard (1993), Multiple Intelligences. Theory Into 
Practice. New York: Basic Books. 

National Research Council (1996), National Science Education 
Standards. Washington, DC: National Academy Press. 



7 




9 




r * 



U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 

Educational Resources Information Center (ERIC) 

REPRODUCTION RELEASE 

(Specific Document) 




TM031969 



I. DOCUMENT IDENTIFICATION: 



Title: 




Author(s): 


Corporate Source: " 




Publication Date: 









II. REPRODUCTION RELEASE: 



In order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announced in the 
monthly abstract journal of the ERIC system, Resources in Education (RIE), are usually made available to users in microfiche, reproduced paper copy, 
and electronic media, and sold through the ERIC Document Reproduction Service (EDRS). Credit is given to the source of each document, and, if 
reproduction release is granted, one of the following notices is affixed to the document. 



If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following three options and sign at the bottom 
of the page. 



The sample sticker shown below will be 
affixed to all Level 1 documents 


The sample sticker shown below will be 
affixed to all Level 2A documents 


The sample sticker shown below will be 
affixed to all Level 2B documents 


PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE, AND IN ELECTRONIC MEDIA 
FOR ERIC COLLECTION SUBSCRIBERS ONLY, 
HAS BEEN GRANTED BY 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE ONLY HAS BEEN GRANTED BY 


A® 




A® 




A® 


cfi' 










TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 


1 




2A 




2B 


Level 1 

T 


Level 2A 

r 


Level 2B 

T 


Ri 


□ 


□ 



Check here for Level 1 release, permitting 
reproduction and dissemination in microfiche or other 
ERIC archival media (e g., electronic) and paper 
copy. 



Check here for Level 2A release, permitting 
reproduction and dissemination in microfiche and in 
electronic media for ERIC archival collection 
subscribers only 



Check here for Level 2B release, permitting 
reproduction and dissemination in microfiche only 



Documents will be processed as indicated provided reproduction quality permits. 

If permission to reproduce is granted, but no box is checked, documents will be processed at Level l. 



Sign 

here,-* 

n/ease 

ERIC 



/ hereby grant to the Educational Resources Inforwetion Center (ERIC) nonexclusive permission to reproduce end disseminete this document 
es indicated ebove. Reproduction from the ERIC microfiche or electronic medie by, persons other then ERIC employees end its system 
contractors requires permission from the copyright holder. Exception is made for non-profit reproduction by libraries end other service egencies 
to satisfy information needs of educators in response to discrete inquiries. 



Signature: 






Organization/Address: 




marlow ediolp 
truiwan^state university 

K'RKSVILLE, mo 63501 





7EZZT £3;^r. FroP' 


j 








E-Mail Address: 





(over) 




III. DcMjHeNT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE): 

If permission to reproduce is not granted to ERIC, or, if you wish ERIC to cite the availability of the document from another source, please 
provide the following information regarding the availability of the document. (ERIC will not announce a document unless it is publicly 
available, and a dependable source can be specified. Contributors should also be aware that ERIC selection criteria are significantly more 
stringent for documents that cannot be made available through EDRS.) 




IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER: 

If the right to grant this reproduction release is held by someone other than the addressee, please provide the appropriate name and 
address: 




V. WHERE TO SEND THIS FORM: 



Send this form to the following ERIC Clearinghouse' 

ERIC CLE ARlmjHOlJ SE ON ASSESSMENT AND EVALUATION 

UNIVERSITY OF MARYLAND 

1129 SHRIVER LAB 

COLLEGE PARK, MD 207^2 

ATTN: ACQUISITIONS 



coi^buted) to' iCi,ed by ,he ERIC Fadlity ' ° r * mak ' n9 an unsolidted con,ribution to ERIC, return this form (and the document being 



ERIC Processing and Reference Facility 
4483 -A Forbes Boulevard 
Lanham. Maryland 20706 

Telephone: 301-552-4200 
Toll Free: 800-799-3742 
FAX: 301-552-4700 
e-mail: ericfac@ineted.gov 
WWW: http://ericfac.piccard.c3c.com 



EFF-088 (Rev. 2/2000) 



